On Local Linear Convergence Rate of Iterative Hard Thresholding for Matrix Completion

Iterative hard thresholding (IHT) has gained in popularity over the past decades for large-scale optimization. However, the convergence properties of this method have only been explored recently in non-convex settings. Existing works in matrix completion provide global convergence guarantees for IHT. To do so, they rely on standard assumptions such as incoherence property and uniform sampling. While such analysis provides a global upper bound on the linear convergence rate, it does not describe the actual performance of IHT in practice. In this paper, we provide a novel insight into the local convergence of IHT for matrix completion. We uncover the exact asymptotic linear rate of IHT in closed-form and identify the region in which the algorithm is guaranteed to converge. Furthermore, we utilize random matrix theory to study the linear rate of convergence of IHT for large-scale matrix completion. We find that asymptotically, the rate can be expressed explicitly in terms of the relative rank and the sampling rate. Finally, we present numerical results to verify our theoretical analysis.